How do people react to AI failure? Automation bias, algorithmic aversion, and perceived controllability

Abstract AI can make mistakes and cause unfavorable consequences. It is important to know how people react such AI-driven negative consequences subsequently evaluate the fairness of AI’s decisions. This study theorizes empirically tests two psychological mechanisms that explain process: (a) heuristic expectations consistent performance (automation bias) subsequent frustration unfulfilled (algorithmic aversion) (b) perceptions controllability over results. Our findings from experimental studies reveal these work in an opposite direction. First, participants tend display more sensitive responses inconsistent thus punitive assessments decision fairness, when compared human experts. Second, as perceive has less control outcomes than experts, they are tolerant their AI.